The present invention relates to an image processing technique for reducing a blur component included in an image with image restoration.
Images obtained by capturing an object through optical apparatuses such as digital cameras and interchangeable lenses include a blur component as an image degradation component caused by various aberrations of an image capturing optical system (hereinafter simply referred to as “an optical system”), such as spherical aberration, comatic aberration, field curvature and astigmatism. Such a blur component is generated because a light flux emitted from one point of an object forms an image with some divergence on an image pickup surface, the light flux being normally converged at one point if there is no aberration or diffraction.
The blur component herein is optically expressed as a point spread function (PSF), which is caused by influence of the aberrations of the optical system and is different from blur caused by defocusing. Moreover, color blur in a color image caused due to longitudinal chromatic aberration, chromatic spherical aberration or chromatic comatic aberration of the optical system can be said to be a difference between blurring degrees of respective light wavelengths. In addition, horizontal color shift caused by chromatic aberration of magnification of the optical system can be said to be position shift or phase shift of color light components caused by differences of image capturing magnifications for the respective color light components.
An optical transfer function (OTF) obtained by performing Fourier transform on the point spread function (PSF) is frequency component information of aberration, which is expressed by complex number. An absolute value of the optical transfer function (OTF), that is, an amplitude component is called a modulation transfer function (MTF), and a phase component is called a phase transfer function (PTF). The MTF and PTF are respectively a frequency characteristic of the amplitude component and a frequency characteristic of the phase component of image degradation caused by the aberration. The phase component is herein shown as a phase angle by the following expression where Re(OTF) and Im(OTF) respectively represent a real part and an imaginary part of the OTF.PTF=tan−1(Im(OTF)/Re(OTF))
Thus, the optical transfer function (OTF) of the optical system degrades the amplitude component and the phase component of the image. Respective points of the object in the degraded image are asymmetrically blurred like comatic aberration.
Moreover, the chromatic aberration of magnification is generated in an image pickup apparatus that captures, according to its spectral characteristics, color components such as R, G and B whose image-forming positions are mutually shifted due to differences of image-forming magnifications of its optical system for respective light wavelengths. Therefore, not only the shift of the image-forming positions among the color components is generated, but also shift of image-forming positions among wavelengths in each color component, that is, image spread due to the phase shift is generated.
There is known a method that corrects (reduces) degradation in amplitude (MTF) and degradation in phase (PTF) using information on an optical transfer function (OTF) of an optical system. This method is called image restoration or image reconstruction. Hereinafter, a process that corrects such image degradation using the optical transfer function (OTF) of the optical system is referred to as “an image restoration process” or more simply as “a restoration process”.
Actually manufactured optical apparatuses have individual variabilities in lens shape, lens holding mechanism and lens driving mechanism caused by manufacturing errors. Such an individual variability influences the optical transfer function (OTF) of the optical apparatus. Thus, in order to more accurately perform the image restoration process according also to the manufacturing errors, it is desirable to produce an image restoration filter for the image restoration process based on the optical transfer function (OTF) of each individual of the optical apparatuses.
Japanese Patent Laid-Open No. 2008-85697 discloses an image restoration method using, in an actual image restoration process, a specific image restoration filter that could provide a certain image restoration evaluation level for a predetermined degraded image among plural prestored image restoration filters. This method enables image restoration according to the above-mentioned individual variabilities of the optical apparatuses caused by the manufacturing errors.
However, the method disclosed in Japanese Patent Laid-Open No. 2008-85697 requires trials of the image restoration processes using the plural image restoration filters, which needs a long process time until a restored image is finally output.
Moreover, a method for enabling the image restoration according to the individual variabilities can be employed, as a different method from the method disclosed in Japanese Patent Laid-Open No. 2008-85697, which measures the optical transfer function of each individual of the optical apparatuses and prepares the image restoration filters for the respective individuals. However, preparing the image restoration filters for the respective individuals of the optical apparatuses in one system performing the image restoration process on those individuals extremely increases an amount of prepared data, which is unrealistic. The image restoration filter is a filter to ideally correct spread of the PSF to one point and therefore is constituted by a two-dimensional and asymmetric data grope, which easily increases the prepared data amount.